Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support of transposed convolutions through pixel padding #835

Merged
merged 33 commits into from
Jan 4, 2024
Merged

Support of transposed convolutions through pixel padding #835

merged 33 commits into from
Jan 4, 2024

Conversation

hleblevec
Copy link
Contributor

@hleblevec hleblevec commented Jun 13, 2023 •
edited
Loading

Gradually adding classes and functions to support the "pixel padding" implementation of transposed convolutions.

  • Custom_op for PixelPadding nodes
  • Test bench to validate the full deconvolution (PixelPadding +Conv) [Ongoing]
  • InferPixelPadding transformation to convert ONNX ConvTranspose node to FMPadding_Pixel + (optional) FMPadding_Batch + ConvInpGen + MVAU
  • Test bench using InferPixelPadding transformation instead of manual graph creation.

@auphelia auphelia marked this pull request as draft June 21, 2023 14:53
hleblevec and others added 17 commits June 28, 2023 09:58
@hleblevec
Changing test to use ConvTranspose from standard ONNX as a reference …
bad0613 
…model for comparison
@hleblevec
Creating (ConvTranspose -> FMPixelPadding + ConvInpGen + MVAU) transf…
3db0e2c 
…ormation
@hleblevec
Swapping weights tensors dimension to fit ususal transposed conv repr…
eb5f89e 
…esentation
@hleblevec
[PixelPaddingDeconv] Updating test to use the InferPixelPaddingDeconv…
73037cb 
… transformation. Validating all tests including stitched IP rtlsim.
@auphelia
Merge remote-tracking branch 'upstream/dev' into feature/pixelpadding
5942839
@auphelia
[Linting] Run pre-commit on files
a4c15df
[MoveScalarMulPastConvTranspose] Creating new streamlining transforma…
c024c39 
…tion to push scalar Mul nodes past ConvTranspose nodes, with associated unit test
@hleblevec
Updating QONNX qnd FINN-HLSIB repo versions to match dependencies
7486217
@hleblevec
Updating QONNX qnd FINN-HLSIB repo versions to match dependencies
9a411ec
@hleblevec
[InferPixelPaddingDeconv] Adding safety check to verify that groups=1…
b368ee0 
…, as depthwise deconvolution is not yet supported
@hleblevec
Merging dev
bdd8d4b
@hleblevec
Updating InferPixelPaddingDeconv to just do the lowering and not the …
457400b 
…full conversion to HLS layers
@hleblevec
Updating QONNX commit has
aba05c5
@auphelia
Merge remote-tracking branch 'upstream/dev' into feature/pixelpadding
75212fc
@hleblevec
Rewriting the test script to account for the change in InferConvInpGe…
d454d60 
…n. Adding support for FMPadding_Pixel in SetFolding
@hleblevec
Merge branch 'feature/pixelpadding' of https://github.com/hleblevec/f…
892c73e 
…inn-fork into feature/pixelpadding
@hleblevec
Correcting typo and removing unecessary RTL option from InferPixelPad…
5a5f078 
…dingDeconv.
@hleblevec hleblevec marked this pull request as ready for review October 20, 2023 12:31
@auphelia auphelia self-requested a review October 20, 2023 12:37
auphelia
auphelia reviewed Jan 3, 2024
View reviewed changes
Copy link
Collaborator

@auphelia auphelia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hleblevec ,
I've cleaned up a few things and adjusted the new test cases. Since the Brevitas version was changed as well, there is a few additional tests that will need to succeed before this will be merged.

@hleblevec
Copy link
Contributor Author

Thanks @auphelia for taking the time to review this and sorry if there were things I forgot. I don't know which tests you are talking about but don't hesitate to ask if you need help running them.

@auphelia auphelia merged commit e9985e6 into Xilinx:dev Jan 4, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@auphelia auphelia auphelia left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hleblevec @auphelia